Model-Based Statistical Testing of a Cluster Utility

As High Performance Computing becomes more collaborative, software certification practices are needed to quantify the credibility of shared applications. To demonstrate quantitative certification testing, Model-Based Statistical Testing (MBST) was applied to cexec, a cluster control utility developed in the Network and Cluster Computing Group of Oak Ridge National Laboratory. MBST involves generation of test cases from a usage model. The test results are then analyzed statistically to measure software reliability. The population of cexec uses was modeled in terms of input selection choices. The J Usage Model Builder Library (JUMBL) provided the capability to generate test cases directly as Python scripts. Additional Python functions and shell scripts were written to complete a test automation framework. The resulting certification capability employs two large test suites. One consists of “weighted” test cases to provide an intensive fault detection capability, while the other consists of random test cases to provide a statistically meaningful assessment of reliability.